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Abstract 


The purpose of this study was to explain the variability in data collected from a single-case 
design study and to identify predictors of communicative outcomes for children with 
developmental delays or disabilities (n = 4). Using SAS® University Edition, we fit multilevel 
models with time nested within children. Children’s level of baseline communication and 
teachers’ frequency of strategy use when directed at the children predicted their outcomes. These 
results indicate that children’s initial level of communication predicted their communicative 
outcomes and also that positive associations exist between teachers’ implementation of evidence- 
based communication strategies when they are directed toward children with disabilities and the 
children’s communicative outcomes. Implications for research and practice are provided. 

Keywords: single-case intervention research, multi-level modeling, inclusive 
environments, professional development, communication development 
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Explaining variance and identifying predictors of children’s communication via a 
multilevel model of single-case design research: Brief report 

Communication and language skills in children’s early years affect learning as well as 
the formation of relationships [1], Further, infant interaction abilities are strongly linked to infant 
social-emotional [2] and cognitive [3] development. In the preschool years, language 
development is also strongly associated with social development [4], with language delays often 
co-existing with behavior problems [5], In addition, early language development is related to 
school readiness [6], to later academic and social success [3], and, more specifically, to reading 
development [7], Indeed, delays and differences in language development in early childhood 
may undermine social relationships and school success for years. 

Despite the critical importance of early language skills, there is variability in the degree 
to which early childhood teachers use strategies that positively influence language development 
[8-12], That is, adults may not always intentionally and deliberately use language strategies that 
promote child initiations and responses and may not always engage in conversations that 
include high-quality language input by adults [13], Thus, a research to practice gap in early 
childhood is widely recognized [14], especially with respect to evidence-based (EB) language 
strategies [1 1], Finding ways for early childhood teachers to gain access to professional 
development (PD) that will allow them to effectively implement EB strategies in language and 
communication skills remains a challenge in the field. 

PD focused on supporting teachers’ implementation of EB strategies should be focused 
on a set of practices, implemented collaboratively with the teacher, grounded in the teachers’ 
practice, and linked to desired outcomes [15], Growing evidence demonstrates that training 
supplemented with coaching is more effective in creating long-lasting change than training 
workshops alone [16, 17], Technology can play an important role in providing effective PD. As 
example, Bug-In-Ear (BIE) technology has been used for over 60 years to support professionals’ 
acquisition of applied skills [1 8], Current research suggests that using this technology to coach 
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teachers increases their implementation of EB strategies and decreases their use of less 
effective practices [19, 20], 

Highlighting the importance of effective PD is the assumption that children’s 
development will be enhanced by teachers’ increased use of EB strategies [21], Studies have 
reported positive associations between teachers’ attendance in PD that resulted in a change in 
their instructional practices and children’s language and literacy outcomes [17, 22], although the 
effects on children may not be immediate or easily detected [22], To better detect children’s 
communicative outcomes as a result of a PD designed to improve educators’ use of EB 
communication strategies, we utilized multilevel modeling to conduct a secondary analysis of 
data collected for a single-case intervention design (SCID) study. The SCID study examined the 
effects of BIE coaching on early childhood teachers’ use of EB strategies and associations 
between the PD and children’s expressive communication for four teacher-child dyads [23], 
Functional relations were determined from the educator-level data by examining data within- 
and between-phases and documenting three demonstrations of an effect [24], Because child 
outcomes were secondary in nature we did not examine causal links between the PD and child 
outcomes, but rather associations between the two variables (see figures 1-4). 

For the original study, the associations between the PD and child-level outcomes were 
examined based upon the six features of visual analysis for SCID research [24, 25]. The main 
challenges in determining an association between the variables were the overlap of data and 
variability in children’s outcomes during the intervention phase. Although there were mean 
improvements for all children from the baseline to the intervention phase, there were drastic 
differences in children’s rate of communication within the intervention phase from one session to 
the next. For example, Dion’s mean rate of communication per min was 3.19 (SD = 1.68), 
whereas Dion’s mean rate of communication was 6.83 per min (SD = 3.66) during the 
intervention phase (see figure 1). These data indicate that the variability in children’s 
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communication during intervention was twice as much as in baseline, which helps justify why we 
sought to explain the variability in children’s outcomes using multilevel modeling. 

Insert figures 1, 2, 3, and 4 about here 

Although visual analysis did not indicate strong relations between the PD and children’s 
outcomes, we decided to use multi-level modeling to further examine outcomes because of the 
burgeoning evidence of the effectiveness of this method in analyzing SCID data [26-28], Thus, 
the purpose of this paper was to use multilevel modeling to explain the variation and identify 
predictors in children’s communicative outcomes. We accounted for the hypothesized 
moderator of teachers’ use of EB communication strategies, as well as contextual factors at the 
child (e.g., initial communication) and classroom (e.g., number of children in the group) level 
that we hypothesized were obscuring the results observed via traditional visual analysis 
procedures. For level-1 predictors, we hypothesized that the number of children in the group 
would be negatively associated with outcomes [29] and the frequency that teachers used the 
strategies with target children would be positively associated [30], For level-2 predictors we 
hypothesized that children with higher initial levels of communication would experience greater 
outcomes than children with lower levels [31], We also predicted better outcomes for children 
with developmental delays (DD) than children with autism spectrum disorders (ASD) [32]. 

Methods 

Participants included four 2-year-old children with DD or ASD who attended inclusive 
early childhood centers in a city in the Southeast. Three children (Dion, Braxton, Doston; 
pseudonyms) were male, one female (Mya; pseudonym). Doston was African-American and the 
other children were bi-racial (Caucasian and African-American). At the start of the study, Dion, 
Braxton, and Doston were significantly delayed in their expressive communication, whereas 
Mya communicated within the low-average range of communication for her age. 

The teachers’ PD intervention focused on communication strategies (imitation, 
expanding language, modeling language, reinforcement, following the child’s lead, offering 
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choices, wait time) that have a strong empirical base of effectiveness in enhancing children’s 
communication. The original study [23] used a multiple-baseline across strategies design that 
was replicated across the four teacher-child dyads. Three strategies were selected by teachers 
and taught one at a time by the first author via 15- to 30-min didactic trainings. Then, BIE 
coaching was provided daily for about 20-min until visual analysis procedures indicated a 
functional relation. At this time, the first author randomized the start-point of the next phase [33], 
The original study [23] provides more information on the PD, EB strategies, and SCID methods. 

Three-min video-recorded observations of small-group play activities were coded by 
trained research staff to identify the frequency with which teachers used their three targeted EB 
communication strategies (e.g., imitation) correctly, whether the strategies were directed at the 
target child, the number of children participating in each play session, and children’s mean 
weighted expressive communication per min (for information on weighting [23]). Kappa (k) was 
calculated to record inter-observer agreement for 30% of each dyad’s sessions, k equaled 1.0 
for identifying whether the communication strategies were directed at the target child and the 
number of children in each group, k ranged from .63 to .73 for teachers’ use of their targeted 
communication strategies and .42 to .70 for children’s communication. The lower range of 
children’s agreement resulted from loud environments during the observation sessions (e.g., 
crying), which made it difficult to hear the children’s verbal communication on video recordings. 

Recent simulation studies have indicated that multilevel modeling is an appropriate 
technique for analyzing SCID data when the primary focus is on the fixed effects associated 
with time-varying predictors, such as our treatment variable and our hypothesized meditator 
(child-directed EB communication strategies). Although the use of multilevel models with SCID 
data leads to biased variance estimates and limited power in the tests for the fixed effects of 
level-2 variables, when the model is correctly specified the fixed effect estimates are unbiased 
and the inferences about them are accurate if restricted maximum likelihood estimation is used 
with the Kenward-Roger small sample size adjustments for inferences [27, 28], Consistent with 
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recommendations from these and other methodologists [34], we used multilevel modeling with 
restricted maximum likelihood (REML) estimation and the Kenward-Roger method of inference. 

Results and Discussion 

The primary purpose of this study was to explain the variation in children’s 
communication using multilevel models to examine the effects of our PD intervention and the 
potential mediation of these effects by the number of child-directed communication strategies 
used correctly. For Dion the rate of communication per min varied across measurement 
occasions from 0 to 18.7 with a mean of 6.0 (SD = 3.7). For Braxton there tended to be fewer 
communications with values ranging from 0 to 6 with a mean of 1.3 (SD = 1 .3), whereas for Mya 
there were more communications with values ranging from 1.3 to 25.7 with a mean of 14.1 (SD 
= 5.9). Finally, for Doston the range was 0 to 1 1 .7 with a mean of 3.8 (SD = 3.1). 

We developed our base model by considering two competing variance structures, and 
three potential control variables [baseline level of communication (M = 4.01, SD = 3.36, grand 
mean centered so M = 0), type of disability (DD = 0; ASD =1), and number of children in the 
group ( M = 4.35, SD = 1 .60, grand mean centered so M = 0)], each of which is unrelated to the 
manipulated treatment variable, but potentially related to the outcome (rate of communication). 
We then added variables to our base model in a series of theory-driven steps [35] to examine 
the effect of our intervention [phase coded baseline phase = 0 and treatment phase = 1] and the 
mediation of this effect through the number of EB communication strategies used with the child 
(M = 3.93, SD = 3.92, grand mean centered so M = 0). 

Initially, we estimated an unconditional model that was complex in terms of the variance 
structure (random variation across participants, within-person variances and autocorrelations 
that were allowed to vary across phases and participants) and compared it to a simpler variance 
structure that assumed a common within-person variance and autocorrelation for all participants 
and phases. The model that assumed heterogeneity in the within-participant variances (AIC = 
723.3, BIC = 712.8) fit statistically significantly better [x 2 (14) = 91.1, p < .05] than the simpler 
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model that assumed homogeneity of the within-participant variances (AIC = 786.4, BIC = 784.6), 
and thus the more complex variance model was adopted (see Model A in Table 1). This finding 
supports outcomes from simulation studies indicating that autocorrelation and heterogeneity of 
variance should be considered when using multilevel modeling for SCID analyses [26-28, 34], 

Next we considered the addition of three potential control variables, one at a time and in 
combination. Baseline communication showed a significant positive association with our 
outcome whether considered individually ((3 = 1.09, p < .0001) or in the context of the other 
potential control variables ((3 = 1.21, p = .0023). No significant relationship was found between 
the outcome and type of disability ((3 = -3.26, p = .58 when considered individually and p = 0.75, 
p = .40 when entered with other controls) or number of children (P = -0.22, p = .17 when 
considered individually and p = -0.19, p = .23 when entered with other controls). Thus, only 
baseline communication was retained as a variable in our base model (see Model B in table 1). 

In Model C (see Table 1), our treatment variable, phase , was added. On average, the 
rate of communication per min was 1 .67 more in the treatment phases than the baseline phases 
(p = .033). We expected that the intervention effect would be mediated by the number of child- 
directed EB communication strategies, more specifically, that the intervention would increase 
the number of child-directed EB communication strategies, which in turn to would lead to more 
child communication. To test this mediation hypothesis, we estimated a multilevel model where 
the number of child-directed EB communication strategies served as the dependent variable 
and found there were more child-directed EB strategies in the treatment phases than the 
baseline phases (P = 1 .46, p < .0001 ). Next we entered the number of child-directed EB 
communication strategies as an additional variable in the multilevel model (see Model D). In this 
model, the effect of the child-directed EB communication strategies was found to be positive 
and statistically significant (P = 0.20, p = .0046), whereas the direct effect of phase was not 
statistically significant (P = 1.47, p = .15). These results are consistent with our mediation 
hypothesis and suggest that the PD increased the teachers’ use of child-directed EB strategies, 
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thereby enhancing children’s communication. This finding was obscured when we examined the 
data via visual analysis procedures and suggests that multilevel model provides an effective 
method for analyzing SCID data and determining an intervention’s direct and indirect effects. 

Insert table 1 about here 

In Model E (see Table 1) we examined whether the effect of the number of child-directed 
EB communication strategies depended on the baseline level of communication by adding the 
interaction of these variables to the model. The positive interaction, which was statistically 
significant (P = 0.07, p = .0041 ), suggests that the positive effect of the number of child-directed 
EB communication strategies (P = 0.36, p < .0001 when level of baseline communication is at 
the mean) is more pronounced when there are higher levels of baseline communication. To get 
a better sense of the strength of this interaction and the effect of child-directed EB strategies, 
consider three children: one that is 1 SD below the mean in baseline communication, one that is 
at the mean, and one that is 1 SD above the mean. If we consider the predicted increase in the 
rate of communication per min as we move from the mean number of EB communication 
strategies to 1 SD above the mean (a change of 3.92 strategies), the predicted increases for the 
three children are 0.43, 1.42, and 2.40 communications per min, respectively. These data 
provide evidence indicating that the EB communication strategies have differential effects on 
children’s communication, namely that the strategies are more beneficial for children with higher 
levels of initial communication. Moreover, future research should examine if there are similarly 
differential effects on children’s communication based upon the specific strategies used. 

The post-hoc nature of the research is the major limitation of the study, because these 
studies provide weaker evidence than a priori developed research studies. Therefore, to 
mitigate the drawbacks of conducting post-hoc analyses, future researchers should plan to 
conduct examinations of SCID data using multilevel modeling to study an intervention’s 
effectiveness and other potential mediators and predictors. Nonetheless, this study provides 
evidence of the benefits of using multilevel models to analyze SCID data. The results of this 
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study indicate that teachers direct more EB strategies toward children with DD and ASD after 
receipt of PD aimed to enhance teachers’ correct use of EB strategies in their classrooms. 
Additionally, the findings provide insight regarding the reason child-level outcomes may be 
difficult to identify in studies examining effects of teachers’ PD [21]. Namely, our results suggest 
that children’s outcomes may vary based upon the frequency with which teachers direct EB 
strategies toward them, as well as the interaction between teachers’ use of EB strategies and 
children’s initial levels of communication. Although these outcomes should not be extended 
beyond the population examined, they provide valuable information to aid future researchers in 
analyzing the relations between teachers’ PD, learning opportunities provided to children with 
disabilities, and children’s developmental outcomes. 
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Table 1 


Model Comparisons 



Model A 

Model B 

Model C 

Model D 

Model E 

FIXED EFFECTS 

Intercept 

5.60 (2.24) 

4.81* 

4.04* 

4.50 (1.06) 

5.08* (0.60) 



(0.40) 

(0.52) 



Baseline Communication 


1.09* 

1.26* 

1.29 (0.22) 

1.25* (0.11) 



(0.12) 

(0.13) 



Phase 



1.67* 

1.47 (0.96) 

0.04 (0.78) 




(0.69) 



Child EBCS 




0.20* (0.07) 

0.36* (0.08) 

Child EBCS * Baseline 





0.07* (0.02) 

Communication 






BETWEEN-CASE VARIANCES 

Intercept 

19.5(16.3) 

0 

0 

1.6 (3.0) 

0 

WITHIN-CASE VARIANCES 

Child 1 Baseline 

21.2 (28.4) 

3.9 (4.0) 

3.0 (2.5) 

5.6 (9.2) 

3.0 (2.5) 

Child 1 Intervention 

13.7 (4.2) 

22.3 (8.3) 

18.3 (6.4) 

13.9 (6.0) 

23.4 (9.2) 

Child 2 Baseline 

1.3 (1.3) 

1.2 (1.2) 

2.5 (3.3) 

7.3 (10.5) 

1.3 (1.5) 

Child 2 Intervention 

2.0 (0.6) 

2.0 (0.6) 

2.1 (0.7) 

2.1 (0.7) 

1.9 (0.6) 

Child 3 Baseline 

46.8 (54.9) 

47.8 (56.1) 

49.5 (57.5) 

45.5 (52.5) 

34.4 (33.8) 

Child 3 Intervention 

41.4 (14.2) 

40.4 (13.8) 

31.2 (9.6) 

25.5 (7.4) 

19.6 (5.6) 

Child 4 Baseline 

2.9 (1.7) 

3.2 (1.8) 

2.9 (1.8) 

2.9 (1.8) 

2.8 (1.6) 

Child 4 Intervention 

11.7 (3.4) 

11.0 (3.0) 

10.9 (2.9) 

9.2 (2.8) 

8.3 (2.3) 

AUTOCORRELA TIONS 

Child 1 Baseline 

.95 (.07) 

.71 (.37) 

.61 (.42) 

.80 (.38) 

.58 (.44) 

Child 1 Intervention 

.17 (.29) 

.53 (.17) 

.42 (.21) 

.24 (.37) 

.58 (.16) 

Child 2 Baseline 

.85 (.17) 

.84 (.18) 

.92 (.11) 

.97 (.04) 

.85 (.19) 

Child 2 Intervention 

.31 (.21) 

.30 (.20) 

.35 (.23) 

.17 (.21) 

.21 (.21) 

Child 3 Baseline 

-.90 (.14) 

-.89 (.14) 

-.88 (.16) 

-.88 (.15) 

-.80 (.24) 

Child 3 Intervention 

.39 (.19) 

.37 (.20) 

.19 (.21) 

.09 (.20) 

-.01 (.20) 

Child 4 Baseline 

-.21 (.50) 

-.06 (.40) 

-.40 (.40) 

-.34 (.42) 

-.26 (.42) 

Child 4 Intervention 

.38 (.17) 

.34 (.17) 

.33 (.17) 

.38 (.18) 

.37 (.17) 

MODEL FIT 

AIC 

723.3 

713.9 

706.5 

703.6 

694.1 

BIC 

712.8 

704.1 

696.7 

693.2 

684.3 


Note. Estimates are provided with standard errors; EBCS = evidence-based communication 
strategies; * = p < .05, significance only indicated for fixed effects. 




Rate of Communication Rate of Communication Rate of Communication Rate of Communication 

per Min per Min per Min per Min 




Routine Sessions 

Figure 2: Braxton’s Weighted Expressive Communication per Min 



Routine Sessions 

Figure 3: Mya's Weighted Expressive Communication per Min 



Figure 4: Doston's Weighted Experssive Communication per Min 


